A MapReduce-supported Data Center Networking Topology
نویسندگان
چکیده
Several novel data center networking (DCN) topologies have been proposed to improve the topological properties of data centers. Unfortunately, it is ignored that whether these topologies are suited for the online applications and infrastructure services running on the corresponding data centers. In this paper, we propose a novel DCN topology, named HyperFat-tree Network (HFN). HFN incarnates the good characteristics of the BCube and Fat-tree topologies, and hence naturally supports the distributed data processing application MapReduce. We then address several challenging issues facing HFN to support MapReduce. Through analysis and simulations, we show that HFN possesses excellent properties and is a viable toplogy for MapReduce.
منابع مشابه
BiDAl: Big Data Analyzer for Cluster Traces
Modern data centers that provide Internet-scale services are stadium-size structures housing tens of thousands of heterogeneous devices (server clusters, networking equipment, power and cooling infrastructures) that must operate continuously and reliably. As part of their operation, these devices produce large amounts of data in the form of event and error logs that are essential not only for i...
متن کاملA MapReduce-supported network structure for data centers
Several novel data center network structures have been proposed to improve the topological properties of data centers. A common characteristic of these structures is that they are designed for supporting general applications and services. Consequently, these structures do not match well with the specific requirements of some dedicated applications. In this paper, we propose a hyper-fat-tree net...
متن کاملF10: A Fault-Tolerant Engineered Network
The data center network is increasingly a cost, reliability and performance bottleneck for cloud computing. Although multi-tree topologies can provide scalable bandwidth and traditional routing algorithms can provide eventual fault tolerance, we argue that recovery speed can be dramatically improved through the co-design of the network topology, routing algorithm and failure detector. We create...
متن کاملPhurti: Application and Network-aware Flow Scheduling for Mapreduce
Traffic for a typical MapReduce job in a datacenter consists of multiple network flows. Traditionally, network resources have been allocated to optimize network-level metrics such as flow completion time or throughput. Some recent schemes propose using application-aware scheduling which can reduce the average job completion time. However, most of them treat the core network as a black box with ...
متن کاملNetwork Map Reduce
Networking data analytics is increasingly used for enhanced network visibility and controllability. We draw the similarities between the Software Defined Networking (SDN) architecture and the MapReduce programming model. Inspired by the similarity, we suggest the necessary data plane innovations to make network data plane devices function as distributed mappers and optionally, reducers. A strea...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2010